AMENDMENT AND RESPONSE UNDER 37 CFR § 1.116 

Serial Number: 10/583,952 
Filing Date: June 20, 2006 

Title: STRUCTURAL CONTENT FILTRATION OF HYPOTHESES IX A COGNITIVE CONTROL FRAMEWORK 

IN THE CLAIMS 

Please amend the claims as follows. 

1. (Previously Presented) A computer-implemented method comprising: 
executing a playback component on a first processor of a first processing system, wherein 

the playback component is adapted to perform image analysis on images displayed on a 
graphical user interface as a result of processing simulated user input data during a playback 
phase of execution of an application program, and wherein the playback component is further 
adapted to analyze captured displayed images recorded during a recording phase of execution of 
the application program on a second processor of a second processing system, and wherein the 
playback component is further adapted to reduce a number of hypotheses for a location of an 
active object in a recorded image generated by the graphical user interface by 

transforming the recorded image and a corresponding playback image; 

determining a sub-image from the transformed recorded image which corresponds to an 
object to be searched for in the transformed playback image; 

determining a subset of points on the transformed playback image which have 
appropriate values for matching the sub-image; and 

filtering hypotheses on the playback image which are more than a selected distance from 
any one of the subset of points. 

2. (Original) The method of claim 1, wherein transforming the recorded image and the 
corresponding playback image comprises changing color values of pixels in the images into at 
least one of black and white pixels. 

3. (Original) The method of claim 1, wherein the object comprises a non-textual object. 

4. (Original) The method of claim 1, wherein the filtered hypotheses are incorrect 
hypotheses for the active object. 
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5. (Original) The method of claim 1, wherein a hypothesis comprises a contour of an 
object on the playback image which corresponds to a contour of an object on the recorded image. 

6. (Original) The method of claim 1, wherein filtering hypotheses comprises rejecting a 
hypothesis if the contour of the hypothesis does not have an intersection with any rectangle 
having a top left corner at a point of the subset of points and a predetermined width and height. 

7. (Original) The method of claim 1, wherein filtering hypotheses comprises rejecting a 
hypothesis if the distance between one of points in the subset of points and a contour of the 
hypothesis is greater than a predetermined distance. 

8. (Currently Amended) An article comprising: a machine accessible readable tangible 
storage medium containing instructions, which when executed, 

execute a playback component on a first processor of a first processing system, wherein 
the playback component is adapted to perform image analysis on images displayed on a 
graphical user interface as a result of processing simulated user input data during a playback 
phase of execution of an application program, and wherein the playback component is further 
adapted to analyze captured displayed images recorded during a recording phase of execution of 
the application program on a second processor of a second processing system, and wherein the 
playback component is further adapted to reduce a number of hypotheses for a location of an 
active object in a recorded image generated by the graphical user interface by 

transforming the recorded image and a corresponding playback image; 

determining a sub-image from the transformed recorded image which corresponds to an 
object to be searched for in the transformed playback image; 

determining a subset of points on the transformed playback image which have 
appropriate values for matching the sub-image; and 

filtering hypotheses on the playback image which are more than a selected distance from 
any one of the subset of points. 
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9. (Original) The article of claim 8, wherein instructions to transform the recorded image 
and the corresponding playback image comprise instructions to change color values of pixels in 
the images into at least one of black and white pixels. 

10. (Original) The article of claim 8, wherein the object comprises a non-textual object. 

11. (Original) The article of claim 8, wherein the filtered hypotheses are incorrect 
hypotheses for the active object. 

12. (Original) The article of claim 8, wherein a hypothesis comprises a contour of an 
object on the playback image which corresponds to a contour of an object on the recorded image. 

13. (Original) The article of claim 8, wherein instructions to filter hypotheses comprise 
instructions to reject a hypothesis if the contour of the hypothesis does not have an intersection 
with any rectangle having a top left corner at a point of the subset of points and a predetermined 
width and height. 

14. (Original) The article of claim 8, wherein instructions to filter hypotheses comprise 
instructions to reject a hypothesis if the distance between one of points in the subset of points 
and a contour of the hypothesis is greater than a predetermined distance. 

15. (Previously Presented) A cognitive control framework system for automatically 
controlling execution of an application program having a graphical user interface comprising: 

a recording component adapted to capture user input data and images displayed by the 
graphical user interface during a recording phase of execution of the application program, and to 
analyze the captured user input data and displayed images to generate an execution scenario 
during the recording phase; and 

a playback component adapted to perform image analysis on images displayed by the 
graphical user interface as a result of processing the simulated user input data during the 
playback phase and captured displayed images from the recording phase, the playback 
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component being adapted to reduce the number of hypotheses for the location of an active object 
in a recorded image generated by the graphical user interface by 

transforming the recorded image and a corresponding playback image; 

determining a sub-image from the transformed recorded image which corresponds to an 
object to be searched for in the transformed playback image; 

determining a subset of points on the transformed playback image which have 
appropriate values for matching the sub-image; and 

filtering hypotheses on the playback image which are more than a selected distance from 
any one of the subset of points. 

16. (Original) The system of claim 15, wherein transforming the recorded image and the 
corresponding playback image comprises changing color values of pixels in the images into at 
least one of black and white pixels. 

17. (Original) The system of claim 15, wherein the object comprises a non-textual object. 

18. (Original) The system of claim 15, wherein the filtered hypotheses are incorrect 
hypotheses for the active object. 

19. (Original) The system of claim 15, wherein a hypothesis comprises a contour of an 
object on the playback image which corresponds to a contour of an object on the recorded image. 

20. (Original) The system of claim 15, wherein filtering hypotheses comprises rejecting a 
hypothesis if the contour of the hypothesis does not have an intersection with any rectangle 
having a top left corner at a point of the subset of points and a predetermined width and height. 

21. (Original) The method of claim 15, wherein filtering hypotheses comprises rejecting a 
hypothesis if the distance between one of points in the subset of points and a contour of the 
hypothesis is greater than a predetermined distance. 



